There is a growing desire in many workplace and other environments to separate the display of a computer system from the application processing parts. In the desired configuration, the display is physically located at the user's desktop, while the processing components of the computer are placed in a central location. The display is then connected to the data processor with some method of communication. Various methods described below attempt to address the significant challenge of transferring a high image bandwidth display content from a centralized data processor across a standard network to a remote display with limited network bandwidth. The simplest method is to periodically send copies of frame buffer information from the data processor. This is impractical for sending a normal resolution display image at a reasonable refresh rate. For example, an image frame of 1280×1024 at 24-bit resolution would take 0.3 seconds of dedicated 100 Base T LAN network bandwidth, making perception-free communications of display information with update rates upwards of 30 frame per second impossible.
An alternative approach is to intercept graphics instructions on the data processor and communicate these across the network. However, this method is intrusive on the host system that requires operating system dependent graphic command routing software. Moreover, a processor and software capable of interpreting the graphics commands is required at the remote user interface which makes the method restrictive in its broad compatibility, adds cost and increases complexity to the remote installation.
In another approach, the data processor compares the previously transferred frame with the current frame and only transfer changes between them. The overall amount of data is thereby decreased, especially in the case of a computer display in which much of the display may be static from frame to frame. However, this approach is expensive to implement because the data processor requires at least two frame buffers. A first frame buffer contains a copy of the previously communicated frame and a second contains the present frame. Given that the previous frame must be compared with the present frame one pixel at a time, an additional temporary delta-buffer, is often requiring this approach is both memory and computationally intensive. There is a noticeable decrease in the performance of applications running on the data processor, especially during applications such as video clips that involve significant screen refresh activity. This is caused by each screen refresh requiring the movement and copying of graphics information between frame buffers across the local system bus of the data processor.
A variation of the frame comparison method reduces the overall data processor memory requirement by segmenting the frame buffer into tiles and maintaining a list of signatures for the tiles. The new frame is tiled and the signature for each new tile is compared with the signature in the list to determine if the tile should be transferred. These tiling and list methods are limited. They require hardware or application-based frame buffers tightly-coupled with the data processing architecture. The copying of pixels and signatures that loads the system bus impacts system performance. Software approaches interrupt the operating system so that background tasks can manage the activity. This further reduces the performance of the data processor. Existing tiled change detect methods are also limited in sophistication. Typically, an operation is only performed when the image has changed, in which case the operation is to send the new image
More general image and video compression methods for transferring computer display images to a remote display are even less suitable. With respect to still image transfer methods, simple progressive image transmission (PIT) methods enable the transmission of still image files across a network. For example, progressive encoding is a standard feature of the JPEG2000 specification and enables the pre-encoding of an image file such that the transfer and display of a reasonable quality image approximation at the client side of the network is prioritized by first displaying the low spatial frequency components of the image, followed by a progressive build to the display of a lossless image over a series of build frames. The advantage of using a PIT method is that the peak bandwidth for the image transfer is lower compared with sending the whole image in a single frame. One shortcoming associated with the non-adaptive nature of many PIT methods is overcome by a variation on simple PIT, termed a “generally adopted PIT regulator” (GAPIT-R). GAPIT-R checks network availability and then encodes and transmits an optimum number of bit planes based on available bandwidth. However, PIT methods are not suitable for computer display applications. One major shortcoming lies in the lack of explicit support for the compound nature of a computer display image comprised of text, pictures, background and high definition icon types, each which has different quality requirements. Another major shortcoming lies in the lack of any encoding efficiencies gained by taking advantage of possible inter-frame commonality of content from one frame update to the next.
Video transmission methods on the other hand are tailored to the transmission of highly dynamic images at fixed frame rates and limited bandwidth. They are relatively insensitive to encode/decode delays and typically use encoding methods unrelated to this discussion. Hybrid variations such as M-JPEG transmit a series of independent JPEG images without applying inter-frame prediction methods typical of other video encoding methods such as MPEG-2, H.264 and others. Consequently, these offer limited compression and tend to consume high network bandwidth, especially in applications that mandate high frame rates. Therefore they remain best suited to specialized applications like broadcast resolution video editing or surveillance systems where the frame rate is low.
In summary, existing still image and video compression techniques are not optimized for the high-quality and low latency encoding requirements of dynamic computer display images. Other methods developed specifically to transfer computer display images require intrusive components or a complex remote display system. This results in higher equipment and maintenance costs and lower performance. Therefore, opportunities remain for a significant improvement in computer display image transfer methods.